Categorical: Contingency tables

1 Goals

1.1 Goals

1.1.1 Goals of this section

  • Contingency tables
    • a.k.a., crosstabs, frequency tables
    • 2-way (2 variables) and 3-way (3 variables)
    • Residuals
  • Chi-square tests of independence
    • Fisher’s Exact Test
    • Conditional, marginal, Simpson’s paradox

1.1.2 Goals of this lecture

  • Contingency tables
  • Study design
  • Measures of relationship
  • Chi-square tests

2 Contingency tables

2.1 Layout and notation

2.1.1 Contingency table

  • Shows the relationship between two (or more) variables
    • Frequency of responses in each category
  • Do you believe in an afterlife?
  Yes No
Female 509 116
Male 398 104

2.1.2 Notation for frequencies

  \(J = 1\) \(J = 2\)  
\(I = 1\) \(n_{11}\) \(n_{12}\) \(\color{white}{n_{1+}}\)
\(I = 2\) \(n_{21}\) \(n_{22}\)  
       

2.1.3 Notation for frequencies

  \(J = 1\) \(J = 2\)  
\(I = 1\) \(n_{11}\) \(n_{12}\) \(n_{1+}\)
\(I = 2\) \(n_{21}\) \(n_{22}\) \(n_{2+}\)
  \(n_{+1}\) \(n_{+2}\) \(n\)
  • \(n_{11}\), \(n_{12}\), \(n_{21}\), \(n_{22}\) are joint frequencies
  • \(n_{1+}\), \(n_{2+}\), \(n_{+1}\), \(n_{+2}\) are marginal frequencies

2.1.4 Notation for probabilities

  \(J = 1\) \(J = 2\)  
\(I = 1\) \(p_{11}\) \(p_{12}\) \(\color{white}{p_{1+}}\)
\(I = 2\) \(p_{21}\) \(p_{22}\)  
       

2.1.5 Notation for probabilities

  \(J = 1\) \(J = 2\)  
\(I = 1\) \(p_{11}\) \(p_{12}\) \(p_{1+}\)
\(I = 2\) \(p_{21}\) \(p_{22}\) \(p_{2+}\)
  \(p_{+1}\) \(p_{+2}\) \(1\)
  • \(p_{11}\), \(p_{12}\), \(p_{21}\), \(p_{22}\) are joint probabilities = \(\frac{n_{ij}}{n}\)
  • \(p_{1+}\), \(p_{2+}\), \(p_{+1}\), \(p_{+2}\) are marginal probabilities = \(\frac{n_{i+}}{n}\) or \(\frac{n_{+j}}{n}\)
  • Note: \(p\) used for sample values, \(\pi\) for corresponding population values

2.1.6 Marginal probability

  • Marginal probability: Probability of \(X\) or \(Y\), collapsing over the other
    • What is the distribution of \(X\), ignoring \(Y\)?
    • What is the distribution of \(Y\), ignoring \(X\)?

2.1.7 Marginal probability: Do you believe in an afterlife?

  • Start with frequencies
  \(Y\) = 1: Yes \(Y\) = 2: No  
\(X = 1\):Female \(n_{11} = \color{red}{509}\) \(n_{12} = \color{red}{116}\) \(n_{1+} = \color{blue}{625}\)
\(X = 2\): Male \(n_{21} = \color{red}{398}\) \(n_{22} = \color{red}{104}\) \(n_{2+} = \color{blue}{502}\)
  \(n_{+1} = \color{blue}{907}\) \(n_{+2} = \color{blue}{220}\) \(n = \color{blue}{1127}\)

2.1.8 Marginal probability: Do you believe in an afterlife?

  • Divide each value by \(n\): The total sample size
  \(Y\) = 1: Yes \(Y\) = 2: No  
\(X = 1\):Female \(p_{11} = \color{red}{0.452}\) \(p_{12} = \color{red}{0.103}\) \(p_{1+} = \color{blue}{0.555}\)
\(X = 2\): Male \(p_{21} = \color{red}{0.353}\) \(p_{22} = \color{red}{0.092}\) \(p_{2+} = \color{blue}{0.445}\)
  \(p_{+1} = \color{blue}{0.805}\) \(p_{+2} = \color{blue}{0.195}\) \(p = \color{blue}{1}\)
  • \(\color{red}{Joint}\) probabilities sum to 1: \(\color{red}{0.452 + 0.353 + 0.103 + 0.092} = 1\)
  • \(\color{blue}{Marginal}\) probabilities for rows sum to 1: \(\color{blue}{0.555 + 0.445} = 1\)
  • \(\color{blue}{Marginal}\) probabilities for columns sum to 1: \(\color{blue}{0.805 + 0.195} = 1\)

2.1.9 Conditional probability

  • Typically, \(X\) is an explanatory variable (predictor)
    • \(Y\) is an outcome variable
  • Conditional probability: Probability of \(Y\) at a given value of \(X\)
    • When \(X = 1\), what is the distribution of \(Y\)?
    • When \(X = 2\), what is the distribution of \(Y\)?
  • Conditional probability is the \(\color{red}{joint~value}\) divided by the \(\color{blue}{marginal~value}\) for that value of \(X\)
    • It is conditional on that value of \(X\)

2.1.10 Conditional probability: Do you believe in an afterlife?

  \(Y\) = 1: Yes \(Y\) = 2: No  
\(X = 1\):Female \(n_{11} = \color{red}{509}\) \(n_{12} = \color{red}{116}\) \(n_{1+} = \color{blue}{625}\)
\(X = 2\): Male \(n_{21} = \color{red}{398}\) \(n_{22} = \color{red}{104}\) \(n_{2+} = \color{blue}{502}\)
  \(n_{+1} = \color{blue}{907}\) \(n_{+2} = \color{blue}{220}\) \(n = \color{blue}{1127}\)

2.1.11 Conditional probability: Do you believe in an afterlife?

  \(Y\) = 1: Yes \(Y\) = 2: No  
\(X = 1\):Female \(n_{11} = \color{red}{509}\) \(n_{12} = \color{red}{116}\) \(n_{1+} = \color{blue}{625}\)
\(X = 2\): Male \(n_{21} = \color{red}{398}\) \(n_{22} = \color{red}{104}\) \(n_{2+} = \color{blue}{502}\)
  \(n_{+1} = \color{blue}{907}\) \(n_{+2} = \color{blue}{220}\) \(n = \color{blue}{1127}\)
  • When \(X = 1\) (female):
    • \(P(Yes) = \frac{\color{red}{509}}{\color{blue}{625}} = 0.8144\)
    • \(P(No) = \frac{\color{red}{116}}{\color{blue}{625}} = 0.1856\)
  • When \(X = 2\) (male):
    • \(P(Yes) = \frac{\color{red}{398}}{\color{blue}{502}} = 0.7928\)
    • \(P(No) = \frac{\color{red}{104}}{\color{blue}{502}} = 0.2072\)

2.2 Sensitivity and specificity

2.2.1 True state vs test result

  Positive Negative  
Diseased 1 0 1
Not diseased 12 87 99
  13 87 100

2.2.2 Sensitivity and specificity

  • Sensitivity
    • Probability of positive for individuals who actually are positive
    • “True positive”
    • Statistical power
  • Specificity
    • Probability of negative for individuals who actually are negative
    • “True negative”
    • 1 - type I error rate

2.2.3 Sensitivity and specificity

  Positive Negative  
Diseased 1 0 1
Not diseased 12 87 99
  13 87 100
  • Sensitivity = \(P(positive~test | diseased) = \frac{n_{11}}{n_{1+}} = \frac{1}{1} = 1.0\)
  • Specificity = \(P(negative~test | not~diseased) = \frac{n_{22}}{n_{2+}} = \frac{87}{99} = 0.88\)

2.2.4 Sensitivity and specificity

  • Ideally, both sensitivity and specificity are high
    • They are probabilities, so near 1
  • Several things are related
    • Sensitivity and specificity
    • Joint probabilities / frequencies
    • Marginal distribution of the \(X\) variable (base rate)

2.2.5 Base rate (with some rounding)

  • Sensitivity = \(0.86\)
  • Specificity = \(0.88\)
  • Base rate = \(\color{OrangeRed}{0.01}\)
  Positive Negative  
Diseased 1 0 1
Not diseased 12 87 99
  13 87 100
  • Sensitivity = \(0.86\)
  • Specificity = \(0.88\)
  • Base rate = \(\color{green}{0.30}\)
  Positive Negative  
Diseased 26 4 30
Not diseased 8 62 70
  32 66 100

3 Study design

3.1 Fixed and random

3.1.1 Designing a study

  • When designing a study
    • What do we constrain vs allow to vary?
    • What comes first?
  • Design leads to analysis
    • Two groups, each with 3 time points
      • Repeated measures ANOVA or mixed model
    • Observe two continuous variables
      • Correlation or linear regression

3.1.2 Fixed vs random

  • The marginal frequencies of a contingency table can be fixed or random

  • Fixed: Chosen by the researcher

    • e.g., Collect data on 50 men and 50 women
  • Random: Vary depending on the sample

    • e.g., Collect data on gender in the sample
  • Note 1: “Fixed” and “random” are kind of (but not exactly) like “manipulated” and “measured”

    • More complicated: “fixed” isn’t always \(X\) – sometimes it’s \(Y\)
  • Note 2: This is just one of many definitions of “fixed vs random”

3.1.3 Why do we care if they’re fixed or random?

  • Probability = random divided by fixed
    • If there are no fixed marginals, we can’t calculate a probability
  • Ratio = random divided by random
    • We can always calculate ratios (e.g., odds ratios…)
  • Basically, what is “fixed” is what you can “condition on”

3.2 Types of study design

3.2.1 Study designs

  • Three study designs with different fixed and random marginals
    • Multinomial (or cross-sectional)
    • Retrospective
    • Prospective (or product binomial)
  • Design of the contingency table determines
    • What is conditioned on
    • What are probabilities
    • How you can talk about the relationship in the table

3.2.2 Overall study design

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo      
Aspirin      
       
  • Relationship between aspirin use (vs placebo) and heart attack
    • \(X\): Aspirin vs placebo
    • \(Y\): Heart attack vs no heart attack

3.2.3 Multinomial

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo     Random
Aspirin     Random
  Random Random Fixed
  • Collect data from \(n\) people
    • Measure aspirin vs placebo, heart attack vs not

3.2.4 Retrospective

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo     Random
Aspirin     Random
  Fixed Fixed  
  • Collect data from specific numbers of heart attack and non patients
    • Measure whether they took aspirin

3.2.5 Prospective

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo     Fixed
Aspirin     Fixed
  Random Random  
  • Collect data from specific number of aspirin and placebo people
    • Measure whether they have a heart attack

3.2.6 Other considerations

  • Cross-sectional vs longitudinal
    • How long does it take to work?
    • Reverse causality
  • Retrospective most useful for rare or unpredictable outcomes
    • Cancer, heart attack, extreme events
  • Measures of association depend on the design
    • Some are only possible with prospective design

4 Measures of relationship

4.1 Design to relationship

4.1.1 Why does design matter?

  • Probability vs ratio
    • What is fixed and what is random?
  • Design of study and contingency table determine
    • What is conditioned on
    • What are probabilities
    • How you can talk about the relationship in the table

4.1.2 Why does design matter?

  • Three ways to talk about relationships in contingency table
    1. Difference in proportion
      • Prospective only
    2. Relative risk
      • Prospective only
    3. Odds or odds ratio
      • Any design

4.1.3 Aspirin treatment for heart disease

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo 189 10845 \(\textbf{11034}\)
Aspirin 104 10933 \(\textbf{11037}\)
  293 21778 22071
  • Relationship between aspirin use (vs placebo) and heart attack
    • \(X\): Aspirin vs placebo
    • \(Y\): Heart attack vs no heart attack
  • Prospective design: The \(X\) marginals are fixed

4.2 Difference in proportion

4.2.1 Difference in proportion: General

  • Null hypothesis
    • \(H_0\): \(\pi_1 = \pi_2\) or \(\pi_1 - \pi_2 = 0\)
  • Observed difference
    • \(p_1 - p_2 = \frac{n_{11}}{n_{1+}} - \frac{n_{21}}{n_{2+}}\)
  • Note: Only for prospective designs (\(X\) marginals fixed)

4.2.2 Difference in proportion: Aspirin example

  • Null hypothesis
    • \(H_0\): \(\pi_{yes|placebo} = \pi_{yes|aspirin}\) or \(\pi_{yes|placebo} - \pi_{yes|aspirin} = 0\)
  • Observed difference
    • \(p_1 = p_{yes|placebo} = \frac{189}{11034} = 0.017\)
    • \(p_2 = p_{yes|aspirin} = \frac{104}{11037} = 0.009\)
    • \(p_1 - p_2 = 0.017 - 0.009 = 0.008\)

4.2.3 Difference in proportion inference: General

  • Standard error for the difference
    • \(SE = \sqrt{\frac{p_1(1 - p_1)}{n_{1+}} + \frac{p_2(1 - p_2)}{n_{2+}}}\)
  • Large sample confidence interval on the difference
    • \((p_1 - p_2) \pm z_{\alpha/2}(SE)\)

4.2.4 Difference in proportion inference: Aspirin example

  • Standard error for the difference
    • \(SE = \sqrt{\frac{p_1(1 - p_1)}{n_{1+}} + \frac{p_2(1 - p_2)}{n_{2+}}} = \sqrt{\frac{0.017(0.983)}{11034} + \frac{0.009(0.991)}{11037}} = 0.0015\)
  • Large sample confidence interval on the difference
    • \((p_1 - p_2) \pm z_{\alpha/2}(SE)\)
    • \((0.017 - 0.009) \pm 1.96(0.0015)\)
    • \([0.0047, 0.0107]\)

4.3 Relative risk

4.3.1 Relative risk: General

  • Null hypothesis
    • \(H_0\): \(\frac{\pi_1}{\pi_2} = 1\)
  • Observed relative risk
    • \(\frac{p_1}{p_2} = \frac{\frac{n_{11}}{n_{1+}}}{\frac{n_{21}}{n_{2+}}}\)
  • Note: Only for prospective designs (\(X\) marginals fixed)

4.3.2 Relative risk: Aspirin example

  • Null hypothesis
    • \(H_0\): \(\frac{\pi_{yes|placebo}}{\pi_{yes|aspirin}} = 1\)
  • Observed relative risk
    • \(p_1 = p_{yes|placebo} = \frac{189}{11034} = 0.017\)
    • \(p_2 = p_{yes|aspirin} = \frac{104}{11037} = 0.009\)
    • \(\frac{p_1}{p_2} = \frac{0.017}{0.009} = 1.818\)

4.3.3 Relative risk inference: General

  • Standard error for the natural log of the relative risk
    • \(SE = \sqrt{\frac{1}{n_{11}} + \frac{1}{n_{21}} + \frac{1}{n_{1+}} + \frac{1}{n_{2+}}}\)
  • Large sample confidence interval on the relative risk
    • Calculate in terms of ln(relative risk)
      • \(ln\left(\frac{p_1}{p_2}\right) \pm z_{\alpha/2}(SE)\)
    • Then exponentiate to convert back to relative risk metric

4.3.4 Relative risk inference: Aspirin example

  • Standard error for the natural log of the relative risk
    • \(SE = \sqrt{\frac{1}{n_{11}} + \frac{1}{n_{21}} + \frac{1}{n_{1+}} + \frac{1}{n_{2+}}} =\)
    • \(\sqrt{\frac{1}{189} + \frac{1}{104} + \frac{1}{11034} + \frac{1}{11037}} =\)
    • \(0.1228\)

4.3.5 Relative risk inference: Aspirin example

  • Large sample confidence interval on the relative risk
    • Calculate in terms of ln(relative risk)
      • \(ln\left(\frac{p_1}{p_2}\right) \pm z_{\alpha/2}(SE)\)
      • \(ln\left(\frac{0.017}{0.009}\right) \pm 1.96(0.1228)\)
      • \([0.3569, 0.8384]\)
    • Then exponentiate to convert back to relative risk metric
      • \([e^{0.3569}, e^{0.8384}]\)
      • \([1.4289, 2.3126]\)

4.3.6 Assymetric confidence limits on relative risk

  • Relative risk = 1.818

  • ln(relative risk) = 0.598

4.3.7 Assymetric confidence limits on relative risk

  • Relative risk = 1.818

  • ln(relative risk) = 0.598

4.4 Odds, odds ratio, logit

4.4.1 Odds

  • Odds = probability of “success” divided by probability of “failure”
    • Ranges from 0 to \(+\infty\)
      • Odds > 1: success is more likely than failure
      • Odds < 1: failure is more likely than success
      • Odds = 1: success and failure are equally likely
  • Odds can be used with any study design

4.4.2 Odds

  • Probability of a “success” divided by probability of “not a success”
    • \(odds = \frac{p}{(1-p)}\)
  • \(P(success) = 0.2\)
  • Odds of success =
    • \(\frac{0.2}{(1 - 0.2)}\) =
    • \(\frac{0.2}{(0.8)}\) =
    • \(0.25\) or \(\frac{1}{4}\)
  • \(P(success) = 0.5\)
  • Odds of success =
    • \(\frac{0.5}{(1 - 0.5)}\) =
    • \(\frac{0.5}{(0.5)}\) =
    • \(1\)
  • \(P(success) = 0.8\)
  • Odds of success =
    • \(\frac{0.8}{(1 - 0.8)}\) =
    • \(\frac{0.8}{(0.2)}\) =
    • \(4\)

4.4.3 Odds ratio

  • Odds ratio is a ratio of odds: Ratio of ratios

\[\theta = \frac{odds_1}{odds_2} = \frac{p_1/(1 - p_1)}{p_2/(1 - p_2)} = \frac{n_{11}/n_{12}}{n_{21}/n_{22}} = \frac{n_{11}n_{22}}{n_{12}n_{21}}\]

  • Ranges from 0 to \(+\infty\)
    • Odds ratio > 1: odds of success is more likely in group 1 than in group 2
    • Odds ratio < 1: odds of failure is more likely in group 1 than in group 2
    • Odds ratio = 1: odds of success in group 1 and group 2 are equal

4.4.4 Odds ratio: General

  • Null hypothesis
    • \(H_0: \frac{\pi_1/(1 - \pi_1)}{\pi_2/(1 - \pi_2)} = 1\)
  • Observed odds ratio
    • \(\theta = \frac{p_1/(1 - p_1)}{p_2/(1 - p_2)} = \frac{n_{11}/n_{12}}{n_{21}/n_{22}} = \frac{n_{11}n_{22}}{n_{12}n_{21}}\)
  • Note: Odds and odds ratios can be used with any study design

4.4.5 Odds ratio: Aspirin example

  • Null hypothesis
    • \(H_0\): \(\frac{\pi_{yes|placebo}/\pi_{no|placebo}}{\pi_{yes|aspirin}/\pi_{no|aspirin}} = 1\)
  • Observed odds ratio
    • \(\theta = \frac{n_{11}/n_{12}}{n_{21}/n_{22}} = \frac{189 / 10845}{104 / 10933} = \frac{0.0174}{0.00951} = 1.832\)
    • Observed ln(odds ratio) = \(ln(1.832) = 0.605\)

4.4.6 Odds ratio inference: General

  • Standard error for the natural log of the odds ratio
    • \(SE = \sqrt{\frac{1}{n_{11}} + \frac{1}{n_{12}} + \frac{1}{n_{21}} + \frac{1}{n_{22}}}\)
  • Large sample confidence interval on natural log of the odds ratio
    • Calculate in terms of ln(odds ratio)
      • \(ln\left(\theta\right) \pm z_{\alpha/2}(SE)\)
    • Then exponentiate to convert back to odds metric

4.4.7 Odds ratio inference: Aspirin example

  • Standard error for the natural log of the odds ratio
    • \(SE = \sqrt{\frac{1}{n_{11}} + \frac{1}{n_{12}} + \frac{1}{n_{21}} + \frac{1}{n_{22}}} =\)
    • \(\sqrt{\frac{1}{189} + \frac{1}{10845} + \frac{1}{104} + \frac{1}{10933}} =\)
    • \(0.1228\)

4.4.8 Odds ratio inference: Aspirin example

  • Large sample confidence interval on the log of the odds ratio
    • Calculate in terms of ln(odds ratio)
      • \(ln\left(\theta\right) \pm z_{\alpha/2}(SE)\)
      • \(ln(1.832) \pm 1.96(0.1228)\)
      • \([0.3647, 0.8462]\)
    • Then exponentiate to convert back to odds ratio metric
      • \([e^{0.3647}, e^{0.8462}]\)
      • \([1.44, 2.331]\)

4.5 Comparisons

4.5.1 Compare: All measures

Measure Study design Calculation Example value
Difference in proportion Prospective \(p_1 - p_2\) 0.008
Relative risk Prospective \(\frac{p_1}{p_2}\) 1.818
Odds ratio Any design \(\frac{p_1/(1 - p_1)}{p_2/(1 - p_2)}\) 1.832
  • Note: \(p_1 = n_{11}/n_{1+}\), \(p_2 = n_{21}/n_{2+}\)

4.5.2 Compare: Relative risk and odds ratio

  • Relative risk = 1.818
  • Odds ratio = 1.832

\[odds~ratio = \frac{p_1/(1 - p_1)}{p_2/(1 - p_2)} = relative~risk \frac{(1 - p_1)}{(1 - p_2)} \]

  • When \(p_1\) and \(p_2\) are both close to 0 or both close to 1
    • Odds ratio and relative risk are very similar
    • In this example, \(p_1 = 0.017\) and \(p_2 = 0.009\)

5 Chi-square tests

5.1 Independence

5.1.1 Independence

  • Contingency table shows the relationship between two variables
  • Example
    • \(X\) = Aspirin vs placebo
    • \(Y\) = Heart attack or not
  • Question: Is there a relationship between \(X\) and \(Y\)?
    • Does knowing something about \(X\) tell you something about \(Y\)?
      • Yes: \(X\) and \(Y\) are related
      • No: \(X\) and \(Y\) are independent

5.1.2 Expected value under independence

  • All statistical tests involve comparing a test statistic to an expected value under the null hypothesis
    • Null hypothesis here: Independence
  • Independence is related to correlation, but stronger
    • Independence between variables \(\rightarrow\) correlation = 0 \(\rightarrow\) covariance = 0
  • Expected value under the null hypothesis
    • Correlation and covariance between two variables are both 0

5.1.3 Expected value of joint frequencies

  • \(E(XY)\) is the joint probability distribution of \(X\) and \(Y\)
    • Cells in the table
  • \(E(X)\) and \(E(Y)\) are their marginal probability distributions
    • Margins of the table

\[E(XY) = E(X) E(Y) - cov(XY)\]

  • The joint frequencies depend on the marginal frequencies as well as the covariance between the variables

5.1.4 Expected value (under independence)

  • If \(X\) and \(Y\) are independent, \(cov(XY) = 0\) and

\[E(XY) = E(X) E(Y)\]

  • If \(X\) and \(Y\) are independent, the joint frequencies are completely determined by the marginal frequencies
    • No covariance between \(X\) and \(Y\)

5.1.5 Expected value under independence

  • Expected joint frequencies: \(\mu_{ij} = \frac{n_{i+} n_{+j}}{n}\)

  • Expected joint frequencies

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo \(\mu_{11} = \frac{n_{1+} n_{+1}}{n}\) \(\mu_{12} = \frac{n_{1+} n_{+2}}{n}\) \(n_{1+}\)
Aspirin \(\mu_{21} = \frac{n_{2+} n_{+1}}{n}\) \(\mu_{22} = \frac{n_{2+} n_{+2}}{n}\) \(n_{2+}\)
  \(n_{+1}\) \(n_{+2}\) \(n\)

5.1.6 Observed values: Aspirin example

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo 189 10845 11034
Aspirin 104 10933 11037
  293 21778 22071

5.1.7 Expected values (independence): Aspirin example

  Heart attack No heart attack \(\color{white}{White text}\)
Placebo 146.48 10887.52 11034
Aspirin 146.52 10890.48 11037
  293 21778 22071
  • For example: \(146.48 = \frac{11034 * 293}{22071}\)

5.2 Chi-square test statistic

5.2.1 Chi-square test statistic

\[\chi^2 = \sum\left(\frac{(n_{ij} - \mu_{ij})^2}{\mu_{ij}}\right)\]

  • Null hypothesis
    • Observed frequencies = expected frequencies
  • If all observed = expected, then \(\chi^2 = 0\)
    • As variables become more related (not independent)
      • Differences between observed and expected increase
      • \(\chi^2\) gets larger

5.2.2 Chi-square test statistic

  • \(\chi^2 = \sum\left(\frac{(n_{ij} - \mu_{ij})^2}{\mu_{ij}}\right) =\)

  • \(\frac{(189 - 146.48)^2}{146.48} + \frac{(10845 - 10887.52)^2}{10887.52} + \frac{(104 - 146.52)^2}{146.52} + \frac{(10933 - 10890.48)^2}{10890.48} =\)

  • \(12.343 + 0.166 + 12.339 + 0.166 = 25.014\)

5.2.3 Chi-square test inference

  • Null hypothesis: Independence
    • Observed frequencies = expected frequencies
  • Degrees of freedom = \((I - 1)(J - 1) = (2 - 1)(2 - 1) = 1\)
    • \(\chi^2_{critical}(1) = 3.86\)
  • \(25.014 > 3.86\)
    • Reject \(H_0\) that the variables are independent
      • Placebo vs aspirin is related to heart attack status

5.3 Summary and Assumptions

5.3.1 Assumptions of chi-square test

  • All expected joint frequencies are at least 5
    • Often incorrect stated as all observed joint frequencies
      • OK to have observed cells < 5
      • Just can’t have expected values < 5
  • Chi-square is a large sample test
    • \(\chi^2\) distribution is continuous
      • Sampling distribution of the test statistic doesn’t start looking like \(\chi^2\) until the sample is quite large

5.3.2 Chi-square tests are everywhere

  • Some are easy to see how they relate to the \((O - E)^2 / E\) format here
    • SEM tests of model fit compare the observed covariance matrix to the expected covariance matrix
  • Some are less obvious
    • Likelihood ratio test
    • \(\chi^2\) tests of regression coefficients

5.3.3 Chi-square test vs odds ratio

  • \(\chi^2\) test and odds ratio
    • Two different ways of assessing the relationship between 2 variables
  • \(\chi^2\) tells you about statistical significance
  • Odds ratio tells you about effect size
  • Use both to get a complete picture of the relationship

5.4 Fisher’s Exact Test

5.4.1 Fisher’s Exact Test

  • “Lady tasting tea” problem from Fisher
    • Can the lady tell if tea or milk was put in first?
    • Very small expected values
  Guess tea Guess milk  
Tea first \(n_{11}\) \(n_{12}\) \(\textbf{4}\)
Milk first \(n_{21}\) \(n_{22}\) \(\textbf{4}\)
  \(\textbf{4}\) \(\textbf{4}\) 8

5.4.2 Fisher’s Exact Test

  • Exact test: Not an approximation via the \(\chi^2\) distribution
    • All the possible ways that observations can be distributed in cells
    • Is observed way less likely than we would expect if variables are independent?
  • “Small sample test” for 2 × 2 contingency tables
    • Often used as alternative when cells < 5
    • But there are additional assumptions

5.4.3 Exact test

  • Flip a coin 2 times
    • First flip: Head or tail
    • Second flip: Head or tail
    • Both flips: HH, HT, TH, TT
  • Probability of HH is 0.25: 1 option out of 4 equally likely options

5.4.4 Fisher’s Exact Test

  • All marginal frequencies are fixed
    • Multinomial: Total fixed
    • Retrospective: Column (outcome) marginals fixed
    • Prospective: Row (predictor) marginals fixed
  • When all marginal frequencies fixed
    • Once you know 1 joint frequency (one cell)
    • You know all the other joint frequencies
    • Test statistic = \(n_{11}\)

5.4.5 Fisher’s Exact Test

  Guess tea Guess milk  
Tea first 3 1 \(\textbf{4}\)
Milk first 1 3 \(\textbf{4}\)
  \(\textbf{4}\) \(\textbf{4}\) 8

6 Summary

6.1 Summary

6.1.1 Summary of this week

  • Contingency tables
    • What they are and how they work
  • Study design
    • Relates to contingency table
    • Relates to measures of relationship
  • Chi-square tests
    • Fisher’s Exact Test

6.1.2 Next week

  • More complicated contingency tables
    • \(2 \times 3\) (and larger) tables
    • 3-way tables: \(2 \times 2 \times 2\) tables
  • Chi-square tests for these table
    • Probing the tables
    • Residuals